.. _Using NeurEco Python API on a Parametric Frequency Sweep problem:

Tutorial: using NeurEco Python API for a Parametric Frequency Sweep problem
============================================================================

The following section uses the test case :std:ref:`FSS test case python`. This test is included in the NeurEco installation package.

Building a Parametric Frequency Sweep model
---------------------------------------------

* Create an empty directory (FSS), extract the :std:ref:`FSS test case python` test case data there. The created directory contains the following files:

  * inputs_train.npy
  * targets_train.npy
  * inputs_valid.npy
  * targets_valid.npy
  * inputs_test.npy
  * targets_test.npy

* Import the required libraries (NeurEco and NumPy):

.. code-block:: python

  from NeurEco import NeurEcoFrequential as Frequential
  import numpy as np
  
* Load the data:

.. code-block:: python

  x_train = np.load('inputs_train.npy')
  y_train = np.load('targets_train.npy')
  x_valid = np.load('inputs_valid.npy')
  y_valid = np.load('targets_valid.npy')
  

* Initialize a NeurEco object to handle the **Parametric Frequency Sweep** problem:

.. code-block:: python

  builder = Frequential.PFS()
  
All the methods provided by the **PFS** class can by viewed by calling the *__methods__* attributes:

.. code-block:: python

  print(builder.__methods__)
  
.. code-block:: text

  *** NeurEco ParametricFrequencySweep methods: ***
  - load
  - save
  - delete
  - evaluate
  - build
  - get_input_count
  - get_out_count
  - load_model_from_checkpoint
  - get_number_of_networks_from_checkpoint
  - export_fmu
  - compute_error
  - perform_input_sweep
  - get_denominator_size

To find out what each parameter of any method does and how to use it, print the doc of the method:

.. code-block:: python

  print(builder.export_fmu.__doc__)
  
.. code-block:: text

  exports a NeurEco model to FMU (Functional Mock-up Interface)
  
  :param fmu_path: string: path where to save the fmu file
  :returt: export_status: int: 0 if export is successful, other int if not
  
* To build the model, run the **build** mothod with the building parameters adjusted to the problem at hand (see :std:ref:`Build NeurEco Parametric Frequency Sweep model with the Python API`):

.. code-block:: python

  builder.build(input_data=x_train, output_data=y_train, 
                # the rest of these parameters are optional
                validation_input_data=x_valid, validation_output_data=y_valid,
                write_model_to="./fssModel/fss.efnn",
                checkpoint_address="./fssModel/fss.checkpoint")
				
* When **build** is called, NeurEco starts the building process:

.. code-block:: text

  *****************Loading the training data******************
  ***********************Building Model***********************
  Validation data will be used.
  00h00m00s info > Running NeurEco Frequential version 3.0.616.0 compiled with MSVC v1928  on Apr  3 2023 @ 16:53:51
  00h00m00s info > Reading Dataset...
  00h00m00s info > Reading data files...
  00h00m00s info > -> Training DataSet successfully imported.
  00h00m00s info > Reading data files...
  00h00m00s info > -> Validation DataSet successfully imported.
  00h00m00s info > Dataset successfully imported
  ...
  
During the build NeurEco saves the intermediate modes to the checkpoint file (defined by the parameter **checkpoint_address**). 
To load and use the intermediate models from this checkpoint: 

* Create a new NeurEco object in which to load the model:

.. code-block:: python

  model = Frequential.PFS()
  
* Determine how many intermediate models the checkpoint contains:

.. code-block:: python

  n = model.get_number_of_networks_from_checkpoint("./fssModel/fss.checkpoint")
  
* Load any intermediate model from the checkpoint using its id (count starts from zero). For this example, at the moment of running the command :math:`n=36`, and the following command loads the intermediate model :math:`n°30 \ (id=29)`:

.. code-block:: python

  model.load_model_from_checkpoint("./fssModel/fss.checkpoint", 29)
  
Now **model** is a valid **PFS** model and can be used as usual.

* Compute the test error of each model saved in the checkpoint

.. code-block:: python

  for i in range(n_models):
    print("Loading and evaluating model", i, "from checkpoint file:")
    model.load_model_from_checkpoint("./fssModel/fss.checkpoint", i)
    neureco_outputs = model.evaluate(x_test)
    l2_error = model.compute_error(neureco_outputs, y_test)
    print("L2 relative error (%):", 100 * l2_error, ", denominator size:", model.get_denominator_size())
	
.. code-block:: text

  Loading and evaluating model 0 from checkpoint file:
  L2 relative error (%): 50.05190646213449 , denominator size: 2
  Loading and evaluating model 1 from checkpoint file:
  L2 relative error (%): 34.454248607229395 , denominator size: 3
  Loading and evaluating model 2 from checkpoint file:
  L2 relative error (%): 31.120229401860765 , denominator size: 4
  Loading and evaluating model 3 from checkpoint file:
  L2 relative error (%): 29.993324285627665 , denominator size: 5
  ...
  L2 relative error (%): 1.442559649332938 , denominator size: 194
  Loading and evaluating model 35 from checkpoint file:
  L2 relative error (%): 1.1768063608433597 , denominator size: 194
  
Evaluate a model
--------------------

* Load the testing data:

.. code-block:: python

  x_test = np.load('inputs_test.npy')
  y_test = np.load('targets_test.npy')
  
* Create a **PFS** object to use for the evaluation: 

.. code-block:: python

  evaluator = Frequential.PFS()
  
.. note::
    It is possible to use the already existing **PFS** object **builder** when the evaluation is done just after the **build**, and **builder** is still available.

* Load the built model:

.. code-block:: python

  load_state = evaluator.load("./fssModel/fss.efnn")

.. note::
    When building or evaluating a NeurEco model, all the used paths do not necessarily need to have an extension when they are passed as parameters to a NeurEco method.

* To extract information from the loaded model, such as the number of inputs and the number of outputs, run:

.. code-block:: python

  n_inputs = evaluator.get_input_count()
  n_outputs = evaluator.get_output_count()
  print("Number of Inputs:", n_inputs)
  print("Number of Outputs:", n_outputs)
  
.. code-block:: text

  Number of Inputs: 6
  Number of Outputs: 2
  
* To evaluate the model on the test data:

.. code-block:: python

  neureco_outputs = evaluator.evaluate(x_test)
  l2_error = evaluator.compute_error(neureco_outputs, y_test)
  print("L2 relative error (%):", 100 * l2_error)
  
.. code-block:: text

  L2 relative error (%): 1.1768063608433597
  
* To save the model in the native NeurEco binary format:

.. code-block:: python

  save_state = evaluator.save("./fssModel/fss_same.efnn")

* To export the model, run one of the following commands (*neureco_embed_pfs* license is required):

.. code-block:: python

  evaluator.export_fmu("./fssModel/fss.fmu")

.. warning::
  Once the NeurEco object is no longer needed, free the memory by deleting the object by calling the **delete** method. For the example above, three objects must be deleted:

  .. code-block:: python

    builder.delete()
    evaluator.delete()
    model.delete()
